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ABSTRACT 


Lung sound is a biological signal that can be used to determine the health 
level of the respiratory tract. Various digital signal processing techniques 
have been developed for automatic classification of lung sounds. Entropy is 
one of the parameters used to measure the biomedical signal complexity. 
Multiscale entropy is introduced to measure the entropy of a signal at a 
particular scale range. Over time, various multiscale entropy techniques have 
been proposed to measure the complexity of biological signals and other 
physical signals. In this paper, some multiscale entropy techniques for lung 
sound classification are compared. The result of the comparison indicates 
that the Multiscale Permutation Entropy (MPE) produces the highest 


Lung sound accuracy of 97.98% for five lung sound datasets. The result was achieved for 
Coar Sere. ained procedure the scale 1-10 producing ten features for each lung sound data. This result is 
Multilayer perceptron better than other seven entropies. Multiscale entropy analysis can improve 
the accuracy of lung sound classification without requiring any features other 
than entropy. 
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1, INTRODUCTION 

Lung sound is one of the biological signals that emerge from the respiration process. Any changes in 
it are generated from foreign bodies or physiological changes in the respiratory tract caused by diseases [1]. 
Differences in patterns of lung sounds can be heard by a doctor using a stethoscope to diagnose diseases [2]. 
Auscultation technique, on the other hand, is very subjective for being dependent upon existing experience 
and expertise of the doctor. 

Various techniques for analyzing lung sounds using computers have been developed. Some of these 
techniques include time-domain analysis techniques, such as statistical analysis based on Hjorth 
descriptor [3], empirical mode decomposition (EMD) [4], or fractal analysis [5]. Several researchers 
proposed to perform lung sound analysis in the frequency domain, such as quantile vector frequency [6] or 
MFCC [7]. Meanwhile, wavelet analysis has been used in [8] for classifying abnormal lung sounds. 

One of the popular biological signal analysis methods is multiscale entropy (MSE), proposed by 
Costa et al. with a coarse-grained procedure for multiscale process and sample entropy for entropy 
measurement [9]. Subsequently, several variants of MSE emerge such as refined-MSE, composite-MSE or 
adaptive MSE (AME) [10]. Other researchers modified their entropy measurement techniques, which result 
in multiscale permutation entropy (MPE) [11], multiscale approximate entropy (MApEN) [12], and 
multiscale fuzzy entropy [13]. In the case of lung sounds, multiscale entropy has been used to analyze the 
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lung sounds of pulmonary alveolitis patients [14]. The results indicated that MSE produces more consistent 
features compared to the spectral method. Similar to biological signal analysis, wavelet entropy is used for 
transformer design [15], while fault diagnosis was analyzed using sample entropy [16] and approximate 
entropy [17]. 

The performance comparison of multiscale entropy for various entropy measurement techniques has 
never previously been done. In this study, we compared the use of multiscale entropy for various entropy 
measurement techniques for lung sound classification. Entropy measurement method in this study included 
Shannon entropy [18], spectral entropy [18], Renyi entropy [19], wavelet entropy [20], approximate 
entropy [21], sample entropy [22], permutation entropy [23], and Tsallis entropy [24]. Those eighth entropies 
measurements are a representation of two types entropy measurement techniques: spectral entropy and 
embedding entropy. Embedding entropy was calculated directly from the signal in the time domain, and the 
spectral entropy was calculated from the signal amplitude in the frequency domain. By this research, we 
obtained the best multiscale entropy techniques for automatic lung sound classification. Next, we could 
recommend the best method for automatic lung sound analysis. From our experiment, MPE produced the 
highest accuracy among those eight entropies. 

This paper is organized as follows. Section 2 explains the experimental method, the lung sound data 
used in this study, entropy measurement methods and multiscale entropy techniques. Meanwhile, the results 
and analysis of the test results described in Section 3. Section 4 contains the conclusions of this paper. 


2. MATERIAL AND METHODS 

Figure | presents the method in this paper. We here used five classes of lung sound data as the input 
data. Meanwhile, in the classification stage, we used multilayer perceptron and three-fold cross-validation. 
The following subsections explain the data, entropy measurement, and classification method in detail. 


Coarse- Ent 
Lung sound -» grained bial hi Classification 
procedure calculation 


Figure |. Block diagram of multiscale entropy in this paper 





2.1. Lung sound data 

The lung sounds were gathered from several sources on the internet [25, 26] and CD companion of 
textbook [27]. All the data were converted in the form of wave files with a sampling frequency of 8000 Hz. 
Furthermore, the data were cut into one respiratory cycle. 

Table 1 presents the detail of lung sound data. Some of the data have been used in a previous study 
[3]. Normal bronchial is a kind of normal lung sound with an expiratory duration relatively longer that 
inspiration phase. It has loud and high-pitch sound with a pause between inspiration and expiration [28]. 
Crackle is a nonmusical and explosive sound that has a short duration. Crackle sound indicates secretion as in 
chronic bronchitis (coarse crackle) or not correlated with secretion as in congestive heart failure (fine 
crackle) [28]. Asthma is a disease that produces a wheezing sound that has a continuous pattern, musical 
sound and dominant frequency more than 400 Hz [29]. Friction rub or pleural rub is associated with pleural 
inflammation or pleural tumors that produce nonmusical, explosive and usually biphasic sound [28]. 
Sometimes, it does not need a stethoscope to listen to the stridor, originating from the larynx or trachea and 
having a dominant frequency > 1000 Hz [8]. 


Table 1. Lung sound data 


Data class Number of data 
Normal bronchial 22 
Crackle 2) 
Asthma 18 
Friction rub 18 
Stridor 20 


On the data, the normalization process is carried out as in (1) and (2). 
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y(n) = x(n) — = OE x(i) (1) 


y(n) = = — with p ==, xi) (2) 


SUN Gi-M)? 


where x(n) is the input signal, N is the length of the input signal, and uy refers to the mean of the input signal. 
Because in (1) mean () has been made as zero, so (2) can be rewritten as (3). 


(n) 
y(n) = —— (3) 
Peover 


The result from (3) is the mean of signal (u) = 0 and standard deviation (o) = 1. The value of o = 1 
would be used in the calculation of sample entropy and approximate entropy. 

Figure 2 shows an example of normal bronchial and its lung sound spectrum frequency, while 
Figure 3 displays the crackle sound example. From Figure 2 and Figure 3, we can see the difference between 
the two types of signals in the time domain and the frequency domain. This is to be distinguished using 
multiscale entropy both spectral entropy and embedded entropy. 
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Figure 2. Normal lung sound and the spectrum 
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Figure 3. Crackle lung sound and the spectrum 


2.2. Multiscale process 
Multiscale entropy was used to view the physiology signal’s entropy at various scales [9]. We used 


the coarse-grained procedure for the multiscale process as in (4). 


(4) 


(t) __lyut 
j= pAiag-arei%i 1 SIS 


a1 


where x(1) is the input signal, t is scale and i is a signal in scale t. In general, the signal i is the output 


signal on a scale t as the average of a number Tt of the input signal x(i). Initially, this method was used in 
conjunction with a sample of entropy to its entropy calculations [9]. In this paper, the coarse-grained 
procedure was used together with the following entropy measurement to extract the features of lung sounds. 


Indonesian J Elec Eng & Comp Sci, Vol. 12, No. 3, December 2018 : 984 — 994 


Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 O 987 


2.3. Entropy measurement 
In this paper, we used eight entropy measurement methods to be combined with the coarse-grained 
procedure. This section explains each entropy measurement method. 


2.3.1. Shannon Entropy 
Shannon entropy (ShEN) is a signal complexity metric. SHEN shows information contain in the 
signal as expressed in (5). 


ShEN = — Qik, pj loge Vi (5) 
where pi is distribution probability of signal, and 7 is a level of the signal. 


2.3.2. Spectral Entropy 
Spectral entropy (SEN) is ShEN measurement in the frequency domain as expressed as (6). 


1 
SEN = —Dip2oPp loge) (6) 


where Pr is the power density of frequency band, and f; and fp are the frequency limit of the signal. The 
power of the signal was normalized so that >} pn = 1. 


2.3.3. Renyi Entropy 
Renyi entropy (REN) is the general form of ShEN [19]. Mathematically REN is expressed as (7). 


1 
Sq = 7a, (092 (diLi pi) 941 (7) 


where q = order of Renyi entropy. Practically, REN is defined for order q = 2. 


2.3.4. Wavelet Entropy 
Wavelet Entropy (WE) is calculated from the energy of each sub-band in the wavelet transformation 
results [20]. WE is expressed as (8). 


WE = — Jiico Pi Np; (8) 
where pi is the relative wavelet energy obtained from (9). 


Ej 


Pi = Et (9) 


where Ei is the energy for i-th resolution, and Et is the total energy. The advantages of WE that it is not 
influenced by noise. WE detects some small changes in non-stationary signals and does not depend on any 
parameters [20]. In this paper, we used Db2 as mother wavelet and decomposition level = 7, as in [30]. 


2.3.5. Approximate Entropy 

Approximate entropy (ApEn) is a signal complexity parameter by measuring the number of 
occurrences of a signal pattern along the signal [21]. If the sequence of the signal along N {u(i):,1 <i < N} 
is given m to form a vector X;” until XV_m41 as in (10). 


X™ = {uG),u(Git+ 1),...,ud@+m-—1)},fori=1,..,N-m+1 (10) 


where m is the length of window which would be compared. For each i < N — m + 1, the defined C;"(r) is 
(N —m + 1)~* multiplied by a number of the the X}” in r from X7”. By defining (11) 


o™r) = (N—-m4+ DUD nce) (11) 


where /n is natural logarithm. 
Pincus defined ApEn as in (12) [21]. 


ApEn(m,r) = lim [@™(r) — BBC) (12) 
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ApEn was estimated using statistic as (13). 
ApEn(m,r,N) = ®™(r) — 6™*7(7) (13) 
Commonly ApEn was calculated using m = 2 and r = 0.2 X standard deviation. 


2.3.6. Sample Entropy 

Sample entropy (SampEn) was proposed by Richman and Moorman to overcome the weakness of 
ApEn [22]. In ApEn, bias is present by self-matches. SampEn is a measure of the probability of a row of m 
data that would be the same as other in a series of the signal with a tolerance of r, which would remain the 
same if the row of m data is increased to m+1. Mathematically SampEn is expressed as (14). 


A) 
B™(r) 





SampEn(m,r) = tim —In (14) 


where A™(r) is the probability of two sequences will match for a number of m+1 samples within tolerance r. 
Meanwhile, B’™(r) is the probability of two sequences that would match for m number of samples within the 
tolerance of r. In both parameters, self-matches was avoided. Furthermore, by making B = {[(N —m— 
1)\(N — m)]/2}B™(r) and A = {[((N — m—1)(N — m)]/2}A"™(r) so SampEn can be expressed as (15). 


SampEn(m,r, N) = — In= (15) 


2.3.7. Permutation Entropy 

Permutation entropy (PE) is the measurement of signal complexity by identifying the probability of 
code sequence in the signal [23]. PE observes the permutation patterns of the different elements in the signal. 
It is expressed as (16). 


PE = — )j=1 Pj log2 p; (16) 


where p; is the relative frequency of possible pattern and n is is permutation order, n > 2. In this paper, we 
used n = 6 asin [11]. 


2.3.8. Tsallis Entropy 

Tsallis entropy (TE) is commonly used to describe the physical behavior of a system [24]. TE 
describes the system with the effect of long-term memory, long-range interaction and multifractal space-time 
constraint [28]. It is non-extensive in which there are two identical systems, then the number of entropy is not 
the same as both added together. TE mathematically is expressed as (17). 


_ 1-Diny P} 
LE = a (17) 


where q is non-extensivity order, pi is discrete probability, and W is the microscopic configuration of the 
system. In this paper we use the order of non-extensivity q = 2 proved to produce the highest 
accuracy in [31]. 


3. RESULTS AND DISCUSSION 
3.1. Measurement Result and Statistics Analysis 

Figure 4 shows the results of the multiscale process using the coarse-grained procedure for the 
normal bronchial and lung sound crackle sound. In general, the signal shape did not change but the data 
length becomes N/t, with N is the original data length, and t is the scale. The value of each sample on the 
scale t was the average value of t sample data from the original data. The coarse-grained procedure reduced 
the variance of the signal, as the scale t increase then the variance will decrease. The decrease in variance 
will change the entropy measurement value. The change of entropy value was utilized as a feature for 
pulmonary sound classification. Figure 5 displays the variance as a function of scale T. 
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Figure 4. Normal bronchial and crackle lung sound for scale 1 — scale 5 
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Figure 5. The variance of each class of data 


Figure 6 - Figure 13 show the average values of the multiscale entropy measurement results in five 
classes of lung sounds. The entropy measurement results tended to be low except Tsallis entropy that 
generated a considerable value but negative value. A small value was influenced by the normalization 
process so that the average value = 0 while the variance value became = |. We could identify that spectral 
entropy and wavelet entropy generated relatively distinct patterns between classes. The result suggested that 
the entropy measurement in the frequency domain produced more distinct characteristics than in the time 
domain. Figure 10 and Figure 11 display that wheeze sound on the SampEn and ApEn have separated values 
to other classes. SampEn is an improvement of ApEn that has some self-matching problems [45]. The 
calculation of both entropies is about the same; hence the resulting entropy also has a similar characteristic. 
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We used the analysis of variance (ANOVA) statistical test to test the separation between the classes 
of feature resulting from the multiscale entropy. The performance of multiscale entropy was determined 
using The ANOVA F-value [32]. A larger F-value indicates a better performance for separating the classes 
tested compared to the lower one. In this study, F-value was measured on the scale of 1-20, 1-15 scale, and so 
on to see the changes in the discrimination performance at different scales. Table 2 shows the results 
ANOVA test. 

Table 2 shows that the multiscale sample entropy produced the largest F-value on the scale of 1-20 
and the scale of 1-15. For the scale of 1-10 and scale of 1-5, the highest F- value was achieved by Renyi 
entropy. For the scale of 1 to the scale of 1-4, the highest F- value was achieved by Tsallis entropy. 
Meanwhile, multiscale Shannon entropy produced the lowest F-value in all conditions. 

It was widely perceived that the F-value decreased as the reduction scale except on multiscale 
Tsallis entropy and multiscale Renyi entropy. In multiscaleTsallis entropy, F-value increased on the scale of 
1-15 and then decreased in the next scale. Meanwhile, in multiscale Renyi entropy, F-value of 1-20 scale 
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increased to a scale of 1-10 then decreased. This indicated that reducing the scale or reducing the number of 
features did not always reduce the rate of separation of data but may increase it. This occurred because of the 
high scale (scale 15 to scale 20) the variance of the signal tended to decrease [10]. 

Thus the entropy generated was not significantly different from other data. The smallest F- value for 
all the multiscale entropy was obtained in scale | or measurement of entropy in the original signal. This 
suggested that the multiscale scheme produces a better performance compared to entropy measurement of on 
a single scale signal only. 


Table 2. F-Values from ANOVA Test 


F value 
MS-Entropy scale scale scale scale scale scale scale scale 
1-20 1-15 1-10 1-5 1-4 1-3 1-2 1 
Multiscale Sample entropy 809.60 709.18 513.69 201.71 145.39 49.66 STAD 24.29 
Multiscale Tsalli's entropy 637.19 638.20 615.42 481.38 425.38 354.12 263.91 148.62 
Multiscale Approximate Entropy SID 1 415.65 = 252.37 92.46 69.20 49.66 33.38 19.51 
Multiscale Permutation entropy 512.33 408.36 334.95 360.62 344.47 275.38 154.25 40.08 
Multiscale Wavelet entropy 400.53 322.37 = 150.48 41.75 32.13 24.57 1522 10.63 
Multiscale Spectral entropy 371.63 322.34 287.77 248.40 210.23 151.73 81.69 31.33 
Multiscale Renyi entropy 360.94 442.09 620.06 512.14 434.54 340.49 233.71 118.13 
Mutliscale Shannon entropy 86.68 57.74 33.48 15.84 12.18 OFS 8.35 3.64 


3.2. Accuracy Testing using Multilayer Perceptro 

To determine the performance of multiscale entropy in the lung sound classification, we used 
multilayer perceptron (MLP) as a classifier. MLP is one variant of artificial neural networks that have 
supervised learning properties. Because the MLP was supervised, then we used the N-fold cross-validation 
(N-fold CV) for the training and testing process [33]. Because the number of data in each class is 18 to 22 
data, then, the N-fold CV used N = 3 so that each dataset would have 6—8 data. Table 3 presents its result. 
Table 3 shows that the multiscale permutation entropy produced the highest accuracy in most of the scales. 
Multiscale Permutation Entropy (MPE) produced the lower accuracy compared to multiscale sample entropy 
(MSE) on the scale of 1—5 and scale of 1-4. PE had several advantages compared with SampEn: PE 
calculated the permutation of the sequence pattern of the signal, while the SampEn calculated the number of 
the same patterns of the signal with a certain tolerance. This implies that the SampEn calculation takes longer 
computation time rather than the PE. Similarities between those two patterns were determined by several 
parameters, such as the n-order permutation on PE, meanwhile, in SampEn, they were determined by the 
similarities tolerance r and length of pattern m. 


Table 3. Accuracy (%) for various multiscale entropies using three-fold cross-validation 


Scale use for entropy measurement 


MS Entropy scale scale scale scale scale scale scale scale 
1-20 1-15 1-10 1-5 1-4 1-3 1-2 1 
Multiscale Sample entropy 85.86 86.87 86.87 86.87 82.83 2nd 69.7 50.51 
Multiscale Tsalli's entropy 88.89 89.9 91.92 79.8 73.74 7273 70.71 69.7 
Multiscale Approximate Entropy 88.89 88.89 83.84 79.8 71.72 65.66 59.6 51.52 
Multiscale Permutation entropy 96.97 96.97 97.98 93.94 92.93 89.9 86.87 59.6 
Multiscale Wavelet entropy 74.75 68.69 67.68 70.71 70.71 2 66.67 43.43 
Multiscale Spectral entropy 93.94 95.96 95.96 90.91 89.9 80.81 75.76 49.49 
Multiscale Renyi entropy 88.89 90.91 93.94 78.79 70.71 74.75 69.7 68.69 
Mutliscale Shannon entropy 83.84 84.85 81.82 76.77 76.77 66.67 58.59 37.37 
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Table 4 summarizes the best results for each of multiscale entropy including a scale that produces 
the highest accuracy, the accuracy of the single scale and average computation time for a single data. Tests 
were performed on a PC with the specification of Intel (R) Core (TM) 13-3220 CPU @ 3.30GHz, 4 GB of 
RAM. 

Table 4 displays that the computational time for the overall technique was acceptable except for the 
multiscale approximate entropy. It took a very long time, so it was less reliable used in real applications. A 
self-matching process caused long computation time occurred in ApEn computation. 


Table 4. Scale for the highest accuracy and computation time 


Multiscale Entropy Best scale ene %) Eee ) cp He as 
Multiscale Sample entropy 1-5 86.87 50.51 27.69 
Multiscale Tsalli's entropy 1-10 91.92 69.7 0.69 
Multiscale Approximate Entropy 1-15 88.89 51.52 2087.47 
Multiscale Permutation entropy 1-10 97.98 59.6 1.61 
Multiscale Wavelet entropy 1-20 74.75 43.43 1.00 
Multiscale Spectral entropy 1-10 95.96 49.49 0.93 
Multiscale Renyi entropy 1-10 93.94 68.69 0.70 
Mutliscale Shannon entropy 1-15 84.85 eye 0.84 


Table 4 shows that the MPE produced the highest accuracy on the scale 1-10 with an accuracy of 
97.98%. Some multiscale entropy only used scale 1-5 to produce the highest accuracy, but still below the 
MPE. Hence, it was not the best choice. Regarding computing time, the MPE required a reasonable 
computing time, i.e., 1.62 s for the single data. Meanwhile, multiscale ApEn took a long computational time. 
This was caused by the calculation of the number of similar patterns of data carried along the sequence of the 
signals and the emergence of self-matching. Multiscale SampEn (MSE) overcame the weakness multiscale 
ApEn, so MSE required shorter computation time than the multiscale ApEn. 

Multiscale SEN produced the highest accuracy after multiscale PE as some signal differences 
between the classes could be seen clearly in the spectral frequency. However, multiscale WE could not 
produce a quite high accuracy for being influenced by the mother wavelet selection, the sampling frequency 
and the level of decomposition. These factors would affect the formed sub-band. The selection of DB2 and 
decomposition level 7 did not produce a high accuracy. Appropriate Wavelet filter and the level of 
decomposition selection will be the next research. 

ShEN, REN, and TE have a similar form. ShEN is a special case of REN and TE. ShEN is REN or 
TE with order g = 1. The selection of order g in REN and TE will produce the different accuracy. The use of 
TE with a different order for lung sound analysis was presented in [31]. The result showed that non- 
extensivity order q = 2 produces the best feature for the lung sound analysis. 

The entropy measurement in the multiscale scheme increased lung sound classification accuracy 
compared with the single scale schemes. Several previous studies used only one entropy values coupled with 
the other features of the lung sound classification. Sample entropy was combined with skewness, kurtosis, 
and lacunarity reported by Mondal et al [34]. The resulting accuracy was 92.86% for normal and abnormal 
lung sound. Meanwhile, kurtosis, mean crossing irregularity and Renyi entropy used in [35]. The results 
obtained the accuracy of 95.1% for the training data and 93.5% for testing data. Tsallis entropy and 25 other 
characteristics used by Morillo et al. for lung sound from congestive obstructive pulmonary disease (COPD) 
patients [36]. Meanwhile, Jin et al. used sample entropy as the features of lung sound extracted from the 
short-time Fourier transform (STFT) [37]. 

The same entropies as in this paper were used in [38]. Single entropy only produced 69.7% of 
accuracy using TE. Composite of seven entropies increased the accuracy became 94.9 %. Computation of 
seven entropies was more complex than one entropy with the multiscale process. Compared with this paper, 
94.9 % was achieved using seven features while 97.9 % was achieved using ten features. 

Overall the studies were conducted on single scale signal. We could see that entropy still needed 
other features for lung sound classification. While on the multiscale entropy, we only used only entropy that 
calculated on a signal with a different scale. Although it was not directly comparable, multiscale entropy 
provided more promising results. Direct comparisons with the same dataset may be made on the research in 
the future. 
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Some multiscale entropies were proposed by some researchers for solving various cases in previous 
studies. Multiscale sample entropy which was more known as multiscale entropy (MSE) was proposed by 
Costa et al [9]. Multiscale permutation entropy was used for quantifying signal complexity proposed by Azis 
and Arif [11]. In this paper, we introduced several multiscale entropies for signal complexity measurement. 
For example multiscale Tsallis entropy, multiscale Renyi entropy, multiscale spectral entropy and multiscale 
wavelet entropy. Multiscale entropy provides better performance for lung sound analysis compared to 
entropy in single scale [38]. 


4. CONCLUSION 

Lung sound is one of the biological signals which have information about the health of the 
respiratory system. To reduce subjectivity in the evaluation of lung sounds t a variety of digital signal 
processing techniques for automatic lung sound classification have been developing. One of the methods 
used in lung sound signal processing is the entropy measurement method. In this research, we compared 
multiscale entropy with some different entropy measurement techniques for lung sound feature extraction. 
The results showed that the multiscale permutation entropy produced the highest accuracy of 97.98% with 
the scale of 1-10. These results are much higher than using Permutation entropy individually on the overall 
signal. Some entropy measurement parameters that could be altered to change such as Tsallis entropy or 
Renyi entropy for a different order of q. Performance of multiscale entropy with various entropy parameters 
can be examined in subsequent research. 
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